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REMARKS 

Reconsideration and allowance of the subject application in view of the foregoing 
amendments and the following remarks is respectfully requested. Claims 1-22 remain pending. 
The specification has been amended to correct minor grammatical errors. No new matter has 
been introduced by these amendments. 

The rejection of claims 1-22 under 35 USC 103 (a) as being unpatentable over Eppler 
(U.S. Patent 6,084,989) in view of Szeliski et al (U.S. Patent 6,157,747) is hereby traversed. 

Applicant's attorney has performed a detailed review of the Eppler reference and has 
been unable to identify where in the reference: 1) a sequence of video frames is extracted, 2) the 
video frames are aligned, or 3) the aligned video frames are used to create a single image, as 
asserted by the examiner. 

First, contrary to the examiner's assertion, patch extractor 17 extrac ts a single image 
patclva25J^ approximately 10,000 lines 

by 20,000 pixels. This is not the same as extrac ting a sequence of video fram^as claimed in 
claim 1. 

Second, with respect to the alignment of video frames, Eppler is directed not to aligning, 
interpolatgd _video frames, but to determining a match between a predicted landmark from a 
geographic information system, i.e., worldwide, seamless, thematic, vector maps input to a 
common mapping tool kit to generate that icon representing a 12 line by 21 pixel image, and an 
actual landmark in an image patch. There is no alignment of interpolated video frames in the 
Eppler reference as claimed in claim 1 . 

The examiner attempts to over come the deficiency of Eppler by combining Szeliski with 
Eppler; however, Szeliski is directed to constractingjni^ 

captured by a camera looking at the same scene but oriented at different angles. That is, Szeliski 
describes a method of stitching together ajjluralit y of images^. This is not the same as aligning an 
extracjt^sequenfce.^ claim 1. 

Further, the Examiner has failed to i dentif y a teac hing, suggestion, or motivation for. 
making the asserted combination o^ppkr^^h^^iski^ In particular, the examiner has failed 
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to identify a teaching, suggestion or motivation for combining the satellite based landmark 
detection system of Eppler with the image mosaic creation method of Szeliski. Both systems 
deal with images; however, Eppler attempts to determine the position of landmarks in satellite 



y images using digitized maps while Szeliski stitches together a plurality of damages to construct a 



continuous mosaic image. A person of ordinary skill in the art would not be motivated to 
combine the teachings of the references. 

A statement that modifications of the prior art to meet the claimed invention would have 
been well within the ordinary skill of the art is not sufficient to establish a prima facie case of 
obviousness without some objective reason to combine the teachings of the references. See 
MPEP 2143.01 quoting Ex parte Levengood, 28 USPQ2d 1300 (Bd. Pat. App. & Inter. 1993). 
The Office Action merely stated that the references can be combined, which Appellants contend 
to the contrary, and does not state any desirability for making the combination. In other words, 
the Office Action failed to supply any objective reasons to combine the applied references. 
Accordingly, the Examiner is requested to identify where, in either reference, there is a teaching 
or suggestion of the asserted combination. 

Third, with respect to the creation of a single image from the aligned video frames, 
Eppler is directed to a system for automatically determining the position of landmarks in 
digitized images from satellite-based images. That is, Eppler uses digitized images to identify 
landmarks in satellite images. There is no creation of a single image using the aligned video 
frames as claimed in claim 1 . 

The examiner attempts to overcome the deficiency of Eppler by combining Szeliski with 
Eppler, as described above. Szeliski is directed to joining together a plurality of images into a 
mosaic and not to combining aligned video frames into a single image as claimed in claim 1 . 

For any of the above reasons, the invention as claimed in claim 1 is patentable over the 
applied combination of references and the rejection should be withdrawn. Claims 2-19 depend, 
either directly or indirectly, from claim land are patentable over the applied combination of 
references for the reasons advanced above with respect claim 1, include further import 
limitations, and the rejection should be withdrawn. Independent claims 20-22 per patentable for 
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reasons similar to those advanced above with respect to claim 1 and the rejection to be 
withdrawn. 

Specific reference is now made to the rejection of dependent claims 2, 5-8„l 1 5 12-15, and 
18-19; these rejections are now addressed ad seriatum. 

With reference to claim 2, the examiner argues that Eppler inherently includes a sequence 
of video frames that are low resolution images because the Eppler system produces a "higher 
resolution display" (column 13, lines 16-22). The examiner is incorrect because the higher 
resolution display referred to is the map used by an operator to create the digitized landmark 
map. In fact, the images used in Eppler, "are very large, and are approximately 10,000 lines by 
20,000 pixels." 

With reference to claims 5, 12, and 19, the examiner argues that extracting the sequence 
of video frames at 30 frames per second is inherent because this rate is standard in 
video/television systems. However, the examiner has not identified any reference teaching 
extraction of a sequence of video frames, nor the extraction of such frames at 30 frames per 
second. Further specifically, neither applied reference teaches using a video/television system 
capturing sequences of video frames at 30 frames per second. 

With reference to claims 6 and 13, the examiner argues that it would be design choice to 
specify a number of video frames in a sequence; however, nowhere in either of the references is 
there a teaching of extracting a sequence of video frames. 

With reference to claims 7 and 14, the examiner argues that the cross correlation 
algorithm of Eppler performs the step of correlating the upsampled video images. The cross 
correlation algorithm in Eppler is used to correlate a landmark mask with a potential landmark in 
an image patch, there is no correlation of a sequence of upsampled video frames. 

With reference to claims 8 and 15, the examiner argues that Eppler averages a pixel 
intensity from each of the upsampled video frames at column 9, lines 31-33. However, the 
examiner cited location of Eppler describes that the average and variants computed for pixel 
values within the landmark perimeter. This is not the same as averaging a pixel intensity from 
each of the upsampled video frames. 
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With reference to claims 11 and 18, the examiners rationale for combining the references 
is not understood. The examiner is requested to clarify his reasoning. 

All objections and rejections having been addressed, it is respectfully submitted that the 
present application should be in condition for allowance and a Notice to that effect is earnestly 
solicited. 

Respectfully submitted, 

LOWE HAUPTMAN GILMAN & BERNER, LLP 




Registration No. 42,940 

USPTO Customer No. 22429 
1700 Diagonal Road, Suite 3 10 
Alexandria, VA 22314 
(703) 684-1111 
(703)518-5499 Facsimile 
Date: June 20, 2002 
RAN/lcw 
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MARKED-UP VERSION SHOWING CHANGES MADE 
IN THE SPECIFICATION : 

Please replace the paragraph beginning on page 8, line 3 with the following: 

Computer system 100 can send messages and receive data, including program code, 
through the network(s), network link 120 and communication interface 118. In the Internet 
example, a server 130 might transmit a requested code for an application program through 
Internet 128, ISP 126, local network 122 and communication interface 118. In accordance with 
the invention, one such downloaded application provides for an image enhancement process used 
in conjunction with a video mosaic process [information discovery and visualization] as 
described herein. 

Please replace the paragraph beginning on page 9, line 3 with the following: 

Refer now to Figures 6A and 6B showing the process of the present invention. At step 
605 the process is started. A step 610, individual frames are extracted from the library. As 
depicted in Figures 2A and 2B, frames 200 and 250 were extracted from the library. At step 615, 
the extracted individual frames are converted to a black and white format. At step 620, edge 
detection is performed by detecting change in [and] intensity from one pixel to adjacent pixels. 
As depicted in Figures 3 A and 3B, the outlines of various structures are detected. At step 625, 
lines are drawn at changes of intensity as depicted in Figure 3. At step 630, regions of interest 
are determined. At step 635, the regions of interest are correlated as explained with reference to 
Figures 7 A and 7B. At step 640, image registration is performed by compensating for 
platform/camera movement. The registration process accounts for motion of the camera by 
determining the frame-to-frame x-y offsets, zoom and rotation. At step 645, frame overlay is 
performed. At step 650, the video mosaic can be viewed. At step 655 the process is ended. 

Please replace the paragraph beginning on page 9, line 18 with the following: 
Refer now to Figure 7 where [were] at step 705 the process is started. At step 7 1 0, the 
centroid region of interest (ROI) is calculated. At step 715, the centroid is compared with 
centroids of the next [to] adjacent frame. At step 720, centroids are selected which are within 
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[ariel] error tolerances. At step 725, there is a foil correlation of average distance from every 
pixel and corresponding structure. At step 730, if the difference is consistent the structure is 
identified as a potential match. At step 735, steps 705-730 are repeated for other structures that 
fall within error tolerance. At step 740, the stored difference calculations are analyzed and select 
matches are based on pixels within structure having the most consistent differences. The 
analysis includes looking for frame-to-frame location as indicated by the difference calculation. 
This consistency will yield x-y translation, rotation and focal length changes. 

Please replace the paragraph beginning on page 10, line 1 with the following: 
Now referring to Figure 8, the process is started at step 805. At step 810, the frame is 
searched for an edge. At step 815, adjacent "on" pixels are followed until an "off pixel is 
detected. At step 820, [it is determined] the locations of the "on " are determined pixels and these 
locations are stored. At step 825, the number of "on" pixels is counted within the structure 
which must exceed a preset threshold. At step 830, the value of the pixels within a designated 
structure is changed to avoid use in future structures. At step 835, steps 805-830 are repeated 
until the entire images in structure detected. At step 840, the process is ended. 

Please replace the paragraph beginning on page 10, line 9 with the following: 

As depicted in Figure 9, five video frames are extracted which were taken at 30 frames a 

second. Thus, there are video frames 910, 920, 930, 940 and 950 which are extracted from a data 

library. The data [date] library can be onboard the UAV. 

Please replace the paragraph beginning on page 10, line 25 with the following: 

Referring now to Figure 14, a flow diagram summarizing the steps of the present 
invention is illustrated. At step 1405, the process is started. At step 1410, a low resolution 
image registration is performed for a sequence of images. At step 1415, each of the images 
upsampled. As depicted in Figures 10 and 11, the images are upsampled by factor 4. Other 
upsampling factors could be used, but the factor of 4 appears to be optimal, yielding the most 
consistent results. Less than 4 could be used, but the idea is to create the highest resolution 
possible. Using a factor greater than 4 will result in diminishing returns, in that the amount of 
memory and processor capacity required will not necessarily produce [product] a sufficiently 
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higher quality image. At step 1420, an x, y registration is performed for the upsampled images. 
At step 1425, the upsampled images are then aligned[. Usingi using a simple correlation 
technique to determine the x-y frame-to-frame offsets. At step 1430, these aligned, upsampled 
images are then combined into a high resolution output image by performing a pixel-by-pixel 
average across all 5 of the upsampled aligned images. At step 1435, the process is ended. 
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